verbs don't reflect temporality at all, and this concept is signalled by adverbs 
of time and some particles which change the status of a condition, occurance, 
or action. Tagalog and Ilokano do it partially through the verb structure (but 
in a different way from English and Spanish), and partially through adverbs of 
time* • 

Description of the Instruments . Each section of the test is accompanied 
by a master chart whose coordinates reflect the relationship between the con- 
cept and the language manifestation for each grammatical item tested* 

Two of the three parts of the test in each langusige have a taped stimulus; 
the stimulus for the third (Structured Response) is supplied by the examiner. 
Of course, native speakers of the language being tested are used to tape, adminis- 
ter and score the test in each language. All of the student's oral responses are 
taped and all items in the test have a visual reference accompanying and reinforc- 
ing each aural stimulus, with the exception of a small "Response-to-Commands" 
section. 

The heart of the system is, of course, the Examiner's Handbook which con- 
tains, among other things, complete instructions for administering and scoring 
the tests. For each section of the test there is a sentence sheet, sample visuals, 
a sample tape script, a vocabulary inventory, a phonology chart, and to facilitate 
scoring, a set of recap sheets . For each sentence of the test, there is a recap 
sheet containing! l) the stimulus sentence; 2) the concept and the language mani- 
festation involved in each item being tested in that sentence; 3) each test item . 
extracted from the sentence and placed in the appropriate box (phonology, mor- 
phology, syntax, or vocabulary); ^) a check-box identifying those sentences which 
exemplify "classroom style"; and 5) at the bottom of the page, a reduced version 
of the visual used for that sentence. 
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A complete test packet contains! 1) the master chart for each section; 
2) a sentence sheet containing all the test sentences of that section { 3) a 
recap sheet for each test item; a vocabulary Inventory; 5) a phonology chart; 
6) a set of visuals; 7) scoring sheets; 8) audio tapes; and 9) an Examiner's 
llandbdok. 

Scoring Procedures * Scoring and diagnostic analysis are accomplished by 
the use of prepared charts which require the scorer to simply circle the num- 
ber .corresponding to the items missed. Scoring is objectivized by a system 
which ignores all but the particular portion of the sentence being tested. Per- 
centage scores of each section of the test are averaged for the final composite 
percentage score and a line graph is supplied which allows the teacher to ob- 
tain an overall visual diagnostic profile by simply circling the number in the 
graph corresponding to the child's score on each concept. 

The scoring procedure we designed supplies, for diagnostic and planning 
purposes, a profile of 1) the student's ability to handle each concept, and 2) 
his areas of strength and weakness. Since all of the tests are comparable in 
their organization and structure and in linguistic content, by administering the 
test in English and in the child's native language, we can compare the profiles 
and determine whether the child's problems are linguistic or perhaps developmental 
in nature. 

Field Testing . The six tests were extensively field -tested in the Seattle 
area and in several districts in Eastern Washington to 1975 and in three Southern 
Idaho school districts in 1976^ A total of some 5,000 tests were administered. 
A partial analysis of the Seattle and Eastern Washington data was completed this 
yesLT. The Idaho data has been collected » compiled and is in the process of analysis* 
However, since this analysis is not yet completed, we will be reporting here only 
on the Washington data. 
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The tests were originally designed with the dual purpose of subjecting 
them to thorough field-testing and of collecting data for the Seattle Public 
Schools as a patrt of a language needs assessment • 

The tests were administered to students in grades K - 6 in ten elementary . ^ 
schoolst These schools were selected, on the one hand, because they were pro- 
spective sites for bilingual education programs, and on the other, because they 
contained concentrations of students whose home language was other than English. 

The total sample in Seattle consisted of 609 students, In grades K - 6, 86 
of whom were a control group of monolingual English speakers selected by random 
sampling in four of these schools. The other 523 comprised an experimental group 
of students identified by the teachers in each of the schools, in a Seattle Schools 
district survey (February, 1975) i as having a home language other than English. 

Tests were administered to the experimental group in English and in their 
home language with an interval of two weeks b'^tween tests insofar as was possible. 
Contamination (in the Listening Comprehension test. Part l) was minimized by 
attempting to administer the English test first to about one-half of the group 
and the home language first to the other half. 

Testing time varied from about twenty-five minutes to thirty-five minutes, 
depending on the language being tested. Tests were usually scored by the examiner 
on the day after they were administered. 

All examiners were native speakers of the language they administered. Each 
of them had at least a B.A. degree, and most were teachers in Seattle schools. 
Over 200 of the 609 English tests were administered by a single examiner working 
full time. Most of the Jlk children in the largest experimental group, the Can- 
tonese, were tested in their home language by two examiners working full time. 
The Spanish testing was done by one examiner, as were most of the Tagalog and 
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most of the Mandarin, and the Ilokano mainly by two examiners. 

During the course of the f ield-testingf we were in constant contact with 
the examiners in order to elicit from them any comments or criticisms. In addi- 
tion > an item analysis was completed by language group, with the dual purpose 
of diagnostic analysis and possible item revision. 

Test data on each student in the sample was collected on prepared data 
sheets by school, by language group and by grade level. Scores were then tabu- 
lated and tables prepared for each of the schools by language and grade level 
which show mean scores, median scores and range of scores at each grade level. 

Scores were re-tabulated by composite groupings and composite tables were 
charted which permit of multiple comparison of scores and other test data. For 
example, in the case of a child in the experimental group at Beacon Hill school 
. who was tested in English and Cantonese, both his Cantonese and English scores 
can be compared to l) those of his peers in his school, 2) those of the entire 
Cantonese group, 3) those of the monolingual English-speaking control group in 
his school, and those of the Composite Control Group. Thus, each of the ex- 
perimental groups has four composite tables at hand for comparison purposes. 

Objectives . The three main objectiveL^ toward which the tests were designed 
were language dominance, language proficiency, and diagnostic analysis. 

As to dominance , since the tests in all six languages are comparable in 
their construction, design, and content, a fairly objective estimate of the child's 
language dominance should be readily available by a comparison of his proficiency 
in each of his languages, as represented by his percentage score in both languages 
and also by an analysis of his diagnostic profiles in eacdh of them. 

We based our determination of dominance on a lO^peroentage-point spread. 
If the child's score in Teigalog, for example, were 90% and his English score 
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verbs don't reflect temporality at all, and this concept is signalled by adverbs 
of time and some particles which phange the status of a condition, occurance, 
or action. Tagalog and Ilokano do it partially through the verb structure (but 
in a different way from English and Spanish), and partially through adverbs of 
time* • 

Description of the Instruments ^ Each section of the test is accompanied 
by a master chart whose coordinates reflect the relationship between the con- 
cept and the language manifestation for each grammatical item tested. 

Two of the three parts of the test in each langusige have a taped stimulus; 
the stimulus for the third (Structured Response) is supplied by the examiner. 
Of course, native speakers of the language being tested are used to tape, adminis- 
ter and score the test in each language. All of the student's oral responses are 
taped and all items in the test have a visual reference accompanying and reinforc- 
ing each aural stimulus, with the exception of a small "Response-to-Commands" 
section. 

The heart of the system is, of course, the Examiner's Handbook which con- 
tains, among other things, complete instructions for administering and scoring 
the tests. For each section of the test there is a sentence sheet, sample visuals, 
a sample tape script, a vocabulary inventory, a phonology chart, and to facilitate 
scoring, a set of recap sheets . For each sentence of the test, there is a recap 
sheet containing^ l) the stimulus sentence j 2) the concept and the language mani- 
festation involved in each item being tested in that sentence; 3) each test item . 
extracted from the sentence and placed in the appropriate box (phonology, mor- 
phology, syntax, or vocabulary); ^) a check-box identifying those sentences which 
exemplify "classroom style"; and 5) at che bottom of the pa^e, a reduced version 
of the visual used for that sentence. 
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recap sheet for each test item; ^) a vocabulary Inventory; 5) a phonology chart; 
6) a set of visuals; 7) scoring sheets; 8) audio tapes; and 9) an Examiner's 
llandbdok. 

Scoring Procedures . Scoring and diagnostic analysis are accomplished by 
the use of prepared charts which require the scorer to simply circle the num- 
ber .corresponding to the items missed. Scoring is objectivized by a system 
which ignores all but the particular portion of the sentence being tested. Per- 
centage scores of each section of the test are averaged for the final composite 
percentage score and a line graph is supplied which allows the teacher to ob- 
tain an overall visual diagnostic profile by simply circling the number in the 
graph corresponding to the child's score on each concept. 

The scoring procedure we designed suppliesi for diagnostic and planning 
purposes, a profile of 1) the student's ability to handle each concept, and 2) 
his areas of strength and weakness. Since all of the tests are comparable in 
their organization and structure and in linguistic content, by administering the 
test in English and in the child's native language, we can compare the profiles 
and determine whether the child's problems are linguistic or perhaps developmental 
in nature . 

Field Testing . The six tests were extensively field -tested in the Seattle 
area and in several districts in Eastern Washington iji 1975 and in three Southern 
Idaho school districts in 1976^ A total of some 5,000 tests were administered. 
A partial analysis of the Seattle and Eastern Washington data was completed this 
yeair. The Idaho data has been collected, compiled and is in the process of analysis* 
However, since this analysis is not yet completed, we will be reporting here only 
on the Washington data. 
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The tests were originally designed with the dual purpose of subjecting 
them to thorough field-testing and of collecting data for the Seattle Public 
Schools as a patrt of a language needs assessments 

The tests were administered to students in grades K - 6 in ten elementary . ^ 
schools. These schools were selected, on the one hand, because they were pro- 
spective sites for bilingual education programs, and on the other, because they 
contained concentrations of students whose home language was other than English. 

The total sample in Seattle consisted of 609 students, In grades K - 6, 86 
of whom were a control group of monolingual English speakers selected by random 
sampling in four of these schools. The other 523 comprised an experimental group 
of students identified by the teachers in each of the schools, in a Seattle Schools 
district survey (February, 1975) i as having a home language other than English. 

Tests were administered to the experimental group in English and in their 
home language with an interval of two weeks b'^tween tests insofar as was possible. 
Contamination (in the Listening Comprehension test. Part l) was minimized by 
attempting to administer the English test first to about one-half of the group 
and the home language first to the other half. 

Testing time varied from about twenty-five minutes to thirty-five minutes, 
depending on the language being tested. Tests were usually scored by the examiner 
on the day after they were administered. 

All examiners were native speakers of the language they administered. Each 
of them had at least a B.A. degree, and most were teachers in Seattle schools. 
Over 200 of the 609 English tests were administered by a single examiner working 
full time. Most of the Jlk children in the largest experimental group, the Can- 
tonese, were tested in their home language by two examiners working full time. 
The Spanish testing was done by one examiner, as were most of the Tagalog and 
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most of the Mandarini and the Ilokano mainly by two examiners. 

During the course of the f ield-testingt we were in constant contact with 
the examiners in order to elicit from them any comments or criticisms. In addi- 
tion > an item analysis was completed by language group, with the dual purpose 
of diagnostic analysis and possible item revision. 

Test data on each student in the sample was collected on prepared data 
sheets by school, by language group and by grade level. Scores were then tabu- 
lated and tables prepared for each of the schools by language and grade level 
which show mean scores, median scores and range of scores at each grade level. 

Scores were re-tabulated by composite groupings and composite tables were 
charted which permit of multiple comparison of scores and other test data. For 
example, in the case of a child in the experimental group at Beacon Hill school 
who was tested in English and Cantonese, both his Cantonese and English scores 
can be compared to l) those of his peers in his school, 2) those of the entire 
Cantonese group, 3) those of the monolingual English-speaking control group in 
his school, and those of the Composite Control Group. Thus, each of the ex- 
perimental groups has four composite tables at hand for comparison purposes. 

Objectives . The three main objectiveL^ toward which the tests were designed 
were language dominance, language proficiency, and diagnostic analysis. 

As to dominance , since the tests in all six languages are comparable in 
their construction, design, and content, a fairly objective estimate of the child* 
language dominance should be readily available by a comparison of his proficiency 
in each of his languages, as represented by his percentage score in both languages 
and also by an analysis of his diagnostic profiles in eacdh of them. 

We based our determination of dominance on a lO^peroentage-point spread. 
If the child's score in Tagalog, for example, were 90% aind his English score 
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onlj' SO%f his Tagalog dominance would be rather obvioust If his English score 
were 80^, however, we would consider him balanced - 90$^ in Tagalog; 80^ in English} 
with a 10-point spread # If a student's score were within a 10-point spread and 
both scores were over 70% (our minimum proficiency level), then we classified 
him as high balanced? if either of his scores was below 70%$ then we considered 
him low balanced • 

As to proficiency , a fairly accurate barometer of language proficiency 
in the English' test was available in the form of a sizeable control group of 
monolingual English speakers, none of whom scored below 70^y^- We had no control 
group available of monolinguals in the home language, but once again the com- 
parability of the tests enabled us to set this same percentage figure (70%) as 
a fairly reliable index of minimal proficiency in the home language as well. 
This was corroborated on a subjective level by the judgement of most examiners 
that students at around the 70% level seemed to be at least minimally functional 
in the language. 

As to dlsigtiostic analysis , the concept profile seems to represent and to 
portray visually an accurate view of the degree of control a student has of the 
structure of the language in question, the peaks and valleys clearly indicating 
his grammatical strengths and weaknesses. And the profile of an entire group 
can be graphed by simply totalling and averaging their scores in each concept 
area and projecting that figure onto the graph. Program and curriculum planning 
can^effected quite simply for the group by examining, in the handbook itself, the 
grammatical manifestations corresponding to the concept areas causing the most 
difficultyt In addition, an overlay of an individual student's profile in English 
on the profile of his performance in his home language can help us in determining 
whether the problems he is having in oral communication are linguistic or perhaps 
developmental in nature. 
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Preliminary Findings » We can "briefly summarize our preliminary findings 
as folic wst 

On test revision, there were surprisingly few items that we found needed 
significant revision^ 

Two of the visuals in the Listening Comprehension section which provided 
some difficulty for a great number of students are in the process of being re- 
vised # 

As to language., only one item in the entire battery of six 'tests - in the 
Ilokano Listening Comprehension section - seemed in our opinion to need a change # 
This is also in the process of being revisedt 

Reports from our examiners in all languages showed a number of alternate 
answers in Part III which need to be added to those we already anticipated In 
the construction of that part of the test. 

On the question of dialect variation, the most significant area here is in 
the CSantonese test. Since a fairly large percentage of the children who took 
the Cantonese test are from homes where Toi San rather than, or in addition to, 
Cantonese is used, the extent of differences between Toi San and Cantonese is in 
the process of being investigated • 

If we look at our largest experimental group, the Cantoneset we find that 
the Cantonese community is, by and large, maintaining the home language, as evi- 
denced by the comparatively large number of diildren who have functional use of 
the home language. Of the total Experimental group, 7^ were able to complete 
the three parts of the Cantonese test, and of this group has as good or bet- 
ter control of Cantonese as they do of English. 

We •re preparing a detailed analysis of their proficiency scores in both 
Cantonese and English, and we're in the process of doing an item analysis in 
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each language in o3rder to determine the areas of linguistic difficulties that 
this group of students is experiencing in each of the two languages. And we'll 
have similar analyses for the Ta/ g.log, Ilokano , Spanish and Mandarin groups. 



diffe3jent from that of the Cantonese group. By and large, the children from 
Spanish-speaking homes are English dominant, and their overall scores indicate 
that they are slightly below the monolingual English speakers when they enter 
school, but fall progressively farther behind through 4th, 5th, and 6th grades. 

We are still in the process of analyzing and summarizing our data. When 
we look at the monolingual English-speaking control group, we find that all of 
these students scored 70% or higher with scores increasing significantly with 
age (overall meant 90%), no doubt reflecting the developmental process of the 
native language. The mean score for kindergarten students was 60%$ with the 
sixth graders almost topping out at 97%* Future diagnostic analysis will focus 
on an error analysis of the items missed most frequently, which should supply 
some indication of the areas of English structure still in the developmental 
stage at each age/grade level, and at the same time suggest important implica- 
tions as to curriculum planning for native English-speaking children in the 
area of English language development. 

Reliability and Validity . A reliability study of the English and Spanish 
tests has just been completed at the University of Idaho (Doyle, 1976) on the 
Eastern Washington sample which consisted of 209 monolingual English speakers 
and 291 bilingual students in grades K - 6. The Kuder-Richardson KR-20 formula 
revealed an overall internal consistency and reliability coefficient of .94 value 
for the English test and .91 for the Spanish test. 

Much work remains to be done, of course, to demonstrate the various kinds 
of validity of the tests and the extent of »,t5ach type. However, evidence to date 
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looks most promisingt As to content validity, various linguists, educatorsi and 
adult native-speakers have examined the tests and have judged the content to be 
highly adequate. Developmental changes reflected in age differentiation and sequen- 
tial patterning of development are apparent in the scores on all six tests, offering 
initial evidence of construct validity. Significant correlation between oral pro- 
f iciciency as measured by the MAT-SEA-CAL- English test and overall achievement as 
measured by the SRA Assessment Survey (Science Research Associates) has been es- 
tablished very recently at the ,01 confidence level by Talbott (1976) at the Uni- 
versity of Idaho, suggesting additional evidence on which to build validity. 

Oral Proficiency and Roading AchicveTnont . V/e also looked into the relation- 
ship between oral proficiency and reading achievement scores of our Seattle sample. 
In order to determine whether there was any significant correlation between spoken 
language proficiency and reading success, we charted the scores made by a group 
of children on the MAT-SEA-CAL English test against the scores made by the same 
children on the MAT reading test (Metropolitan Achievement Test), We were able to 
obtain MAT reading scores (raw and stanine) on 1^7 of the total sample of 152 
fourth and sixth graders who took the MAT-SEA-CAL English test. This included 
both the experimental and control groups. 

The distribution of the scores reveals that no student who scored below 8^ 
on the oral proficiency test was able to score in the fifth stanine in reading. 
That is, 90^ of those with an oral proficiency score of less than 8^ did no better 
in reading than stanines 1, 2, or 3- In the range between 8^ and 90$^ scores in 
oral proficiency, we see the einergence of some few five-stanines, but still no 
sixes. In this range', 70^ of the students are still 'stanine four or below. Only 
in the above-90?S range in oral proficiency do we find appreciable numbers of sta- 
nines five through nine. 
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It would seem, then, that if the fourth and sixth grade student is going 
to have even the possibility of minimal success in reading (i.e. stanine five), 
he would have to have sufficient control of spoken English to score 8^ or above 
on the MAT-SEA-CAL. This corresponds to the mean score of the Seattle English-speak- 
ing m<3nolingual control group at the kindergarten level. The implications for 
educational planning and program development contained in these findings would 
appear to be considerable. This, we believe, is a good thought-provoking notion 
to leave you with. 
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ABSTRACTT 



The MAT-SEA-CAL Instruments for Assessing Language Proficiency 

Joseph H. Matluck Betty Kace^Katluck 

The University of Texas at Austin Seattle Public Schools 



The Multilingual Test Development Project, recently completed in Seattle, 
Washington, has yielded a series of comparable oral proficiency tests in six 
languages designed for use with elementary school children, K - 6. Extensive 
field-testing of the English, Spanish, Cantonese, Mandarin, Tagalog, and Ilokano 
tests was conducted with a research sample of several thousand students, both 
in the metropolitan area of Seattle and in seve^ral of the large migrant centers 
of Eastern Washington and Southern Idaho. This paper describes the project, re- 
ports on the preliminary findings, and focuses on the problems encountered and 
the solutions devised in attempting oral language assessment in a multilingual 
community. Both the findings and the research design will have broad Implica- 
tions for educational planning and program development by school iistricts through- 
out the country and for further research in these areas. 
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